A Theoretical Framework for Studying Random Walk Planning

Random walks are a relatively new component used in several state of the art satisficing planners. Empirical results have been mixed: while the approach clearly outperforms more systematic search methods such as weighted A* on many planning domains, it fails in many others. So far, the explanations for these empirical results have been somewhat ad hoc. This paper proposes a formal framework for comparing the performance of random walk and systematic search methods. Fair homogenous graphs are proposed as a graph class that represents characteristics of the state space of prototypical planning domains, and is simple enough to allow a theoretical analysis of the performance of both random walk and systematic search algorithms. This gives well-founded insights into the relative strength and weaknesses of these approaches. The close relation of the models to some well-known planning domains is shown through simplified but semi-realistic planning domains that fulfill the constraints of the models. One main result is that in contrast to systematic search methods, for which the branching factor plays a decisive role, the performance of random walk methods is determined to a large degree by the Regress Factor, the ratio between the probabilities of progressing towards and regressing away from a goal with an action. The performance of random walk and systematic search methods can be compared by considering both branching and regress factors of a state space. Random Walks in Planning Random walks, which are paths through a search space that follow successive randomized state transitions, are a main building block of prominent search algorithms such as Stochastic Local Search techniques for SAT (Selman, Levesque, and Mitchell 1992; Pham et al. 2008) and Monte Carlo Tree Search in game playing and puzzle solving (Gelly and Silver 2008; Finnsson and Björnsson 2008; Cazenave 2009). Inspired by these methods, several recent satisficing planners also utilize random walk (RW) techniques. Identidem (Coles, Fox, and Smith 2007) performs a hill climbing search that uses random walks to escape from plateaus or saddle points. All visited states are evaluated using a heuristic function. Random walks are biased towards states with Copyright c © 2012, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. lower heuristic value. Roamer (Lu et al. 2011) enhances its best-first search (BFS) with random walks, aiming to escape from search plateaus where the heuristic is uninformative. Arvand (Nakhost and Müller 2009) takes a more radical approach: it relies exclusively on a set of random walks to determine the next state in its local search. For efficiency, it only evaluates the endpoints of those random walks. Arvand also learns to bias its random walks towards more promising actions over time, by using the techniques of Monte Carlo Deadlock Avoidance (MDA) and Monte Carlo with Helpful Actions (MHA). In (Nakhost, Hoffmann, and Müller 2012), the local search of Arvand2 is enhanced by the technique of Smart Restarts, and applied to solving Resource Constrained Planning (RCP) problems. The hybrid Arvand-LS system (Xie, Nakhost, and Müller 2012) combines random walks with a local greedy best first search. Compared to all other tested planners, Arvand2 performs much better in RCP problems (Nakhost, Hoffmann, and Müller 2012), which test the ability of planners in utilizing scarce resources. In IPC domains, RW-based planners tend to excel on domains with many paths to the goal. For example, scaling studies in (Xie, Nakhost, and Müller 2012) show that RW planners can solve much larger problem instances than other state of the art planners in the domains of Transport, Elevators, Openstacks, and Visitall. However, the planners perform poorly in Sokoban, Parking, and Barman, puzzles with a small solution density in the search space. While the success of RW methods in related research areas such as SAT and Monte Carlo Tree Search serves as a good general motivation for trying them in planning, it does not provide an explanation for why RW planners perform well. Previous work has highlighted three main advantages of random walks for planning: • Random walks are more effective than systematic search approaches for escaping from regions where heuristics provide no guidance (Coles, Fox, and Smith 2007; Nakhost and Müller 2009; Lu et al. 2011). • Increased sampling of the search space by random walks adds a beneficial exploration component to balance the exploitation of the heuristic in planners (Nakhost and Müller 2009). • Combined with proper restarting mechanisms, random walks can avoid most of the time wasted by systematic 57 Proceedings of the Fifth Annual Symposium on Combinatorial Search